The Latent Maximum Entropy

Learning Mixture Models with the Latent Maximum Entropy Principle

We present a new approach to estimating mixture models based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes’ maximum entropy principle and from standard maximum likelihood estimation. We demonstrate the LME principle by deriving new algorithms for mixture model estimation, and show how robust new variants of the EM al...

Mixtures of Conditional Maximum Entropy Models

Driven by successes in several application areas, maximum entropy modeling has recently gained considerable popularity. We generalize the standard maximum entropy formulation of classification problems to better handle the case where complex data distributions arise from a mixture of simpler underlying (latent) distributions. We develop a theoretical framework for characterizing data as a mixtu...

Boltzmann Machine Learning with the Latent Maximum Entropy Principle

We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have pro­ posed: the latent maximum entropy principle (LME). LME is different both from Jaynes' maximum entropy principle and from stan­ dard maximum likelihood estimation. We demonstrate the LME principle by deriving new algorithms for Boltzmann machine pa­ rameter estimation, and show h...

Semantic n-gram language modeling with the latent maximum entropy principle

Latent Maximum Entropy Approach for Semantic N-gram Language Modeling